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Program Description 1 Reading Apprenticeship® is an instructional approach that 

intends to help middle school, high school, and community 
college students develop skills and knowledge to improve 
their engagement, fluency, and comprehension of content- 
area materials and texts. To achieve these goals, Reading 
Apprenticeship ® provides a range of professional development 
activities for teachers, as well as an academic literacy curricula 
for students. In both cases, Reading Apprenticeship® calls for 
the teacher to assume the role of expert reader. In this role, 
the teacher models and guides students’ text-based problem 



solving in order to build students’ comprehension strategies. By 
incorporating student/teacher discussions about the process 
of reading into content-area classes, Reading Apprenticeship® 
aims to make the teacher’s and students’ reading processes 
and knowledge visible to others in the classroom, help students 
understand and regulate their own reading processes, and help 
students develop strategies for overcoming obstacles while 
reading and for improving comprehension of texts from core 
academic disciplines. 



Research 2 One study of Reading Apprenticeship® that falls within the 

scope of the Adolescent Literacy review protocol meets What 
Works Clearinghouse (WWC) evidence standards. The study 
included more than 2,000 ninth-grade students who attended 
17 high schools located in 10 school districts across the 
United States. 3 



Based on this study, the WWC considers the extent of evidence 
for Reading Apprenticeship ® on adolescent learners to be small 
for comprehension. No studies that meet WWC evidence stan- 
dards with or without reservations examined the effectiveness of 
Reading Apprenticeship® on adolescent learners in the alphabet- 
ics, reading fluency, or general literacy achievement domains. 



1. The descriptive information for this program was obtained from publicly available sources: the program’s website (http://www.wested.org/ReadingAp- 
prenticeship, downloaded November 2009) and Reading for understanding: A guide to improving reading in middle and high school classrooms 
(Schoenbach, Greenleaf, Cziko, and Hurwitz, 1999). The WWC requests developers to review the program description sections for accuracy from their 
perspective. Further verification of the accuracy of the descriptive information for this program is beyond the scope of this review. The literature search 
reflects documents publicly available by April 2009. 

2. The studies in this report were reviewed using WWC Evidence Standards, Version 2.0 (see the WWC Procedures and Standards Handbook, Chapter III), 
as described in protocol Version 2.0. 

3. The evidence presented in this report is based on available research. Findings and conclusions may change as new research becomes available. 
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Effectiveness Reading Apprenticeship ® was found to have potentially positive effects on comprehension for adolescent learners. 



General literacy 





Alphabetics 


Reading fluency 


Comprehension 


achievement 


Rating of effectiveness 


na 


na 


Potentially positive effects 


na 


Improvement index 4 


na 


na 


Average: +2 percentile points 


na 




na 


na 


Range: -2 to +5 percentile 


na 



points 



na = not applicable 



Additional program Developer and contact 

information Developed by Strategic Literacy Initiative at WestEd, Reading 
Apprenticeship® is distributed by WestEd. Address: 300 Lake- 
side Drive, 25th Floor, Oakland, CA 94612-3534. Email: jbouc@ 
wested.org. Web: http://www.wested.org/ReadingApprentice- 
ship. Telephone: (510) 302-4245. 

Scope of use 

Since 1996, the Reading Apprenticeship ® approach has been 
used across a range of content areas in middle and high school 
classrooms. Since 2005, it also has been used in community col- 
lege classrooms. Reading Apprenticeship ® has been used with 
a wide range of students, including those identified as struggling 
readers, English language learners, special education students, 
and students in honors and advanced placement classes. 

Teaching 

The aim of Reading Apprenticeship® is to improve students’ read- 
ing skills by having them learn how to closely examine both their 
own reading strategies and processes and those of their teacher 
and fellow students. The goal is for students to learn to recognize 
the strategies they have already been using and to develop new 
ones in a way that is directly related to content-area text they 
encounter in class. Small-group and full-classroom conversations 
about, and students’ reflections on, the thinking process being 
used while reading are fundamental components of the program. 



Reading Apprenticeship® calls for teachers to integrate four 
dimensions of classroom life into subject-area teaching through 
conversations about the thinking processes that students and 
teachers use as they read. Per the developer’s website, these are 
the four dimensions: 

Social: This dimension uses students’ interests in social 
interaction to provide a learning environment that reflects 
the diverse perspectives and resources of each individual. It 
involves creating a safe, collaborative environment in which to 
discuss academic texts. 

Personal: This dimension draws on skills used by students in 
out-of-school settings, students’ interest in deepening their 
awareness of the thinking processes used while reading, 
students’ identities as readers, and their purposes and goals 
for reading. 

Cognitive: This dimension involves developing readers’ 
mental processes, including the specific comprehension and 
problem-solving strategies that can be applied to academic 
texts. 

Knowledge-building: This dimension includes identifying and 
expanding the knowledge readers bring to a text, including 
knowledge about word construction, vocabulary, text struc- 
ture, genre, language, and content. 

Reading Apprenticeship ® staff and consultants provide 
professional development and train-the-trainer sessions for 
content-area middle and high school teachers, literacy coaches, 



4. These numbers show the average and range of student-level improvement indices for all findings across the study. 
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Additional program site-level administrators, and district leaders to help them incor- 
information (continued) porate Reading Apprenticeship® in classrooms, schools, and 

districts. Professional development involves participants examin- 
ing their own and their colleagues’ discipline-specific reading 
and students’ reading and writing during training sessions and 
on-site meetings. WestEd also offers The Leadership Institute in 
Reading Apprenticeship®, a two-part, eight-day training institute 
designed to train teams of educators to lead professional devel- 
opment in Reading Apprenticeship®. 

Additionally, the program developers offer a year-long cur- 
riculum for 9th-grade students. The Reading Apprenticeship® 
Academic Literacy curriculum is designed to support students’ 
reading achievement, engagement, and fluency. The course— 
which is organized around three thematic units: (1) Reading Self 
and Society, (2) Reading History, and (3) Reading Science— is 
designed to build students’ motivation, strategic and critical 
reading skills, and ability to construct meaning from academic 
texts. The course focuses students’ attention on how they read 
(a metacognitive process) to help them better understand what 



they read (understanding content). As part of the course, students 
read high-interest, challenging texts; analyze the way words and 
sentences are constructed; and use writing as a tool for learning. 

Cost 

WestEd provides professional development and materials 
to support teachers’ use of Reading Apprenticeship® in the 
classroom. These materials include books, videos, and a cur- 
riculum for a 9th-grade course called Reading Apprenticeship® 
Academic Literacy. 

Standard costs for Reading Apprenticeship® training (for up 
to 40 participants) held at a district-selected location range from 
$15,000 for two days of training up to $50,000 for seven days 
of training. The cost to attend the Reading Apprenticeship® 
Academic Literacy five-day training for the 9th-grade course 
ranges from $3,000 (site based) to $3,200 (San Francisco) per 
participant. 

Additional information on costs of training materials and 
workshops is available online (http://www.wested.org/sli). 



Research Twenty-seven studies reviewed by the WWC investigated the 

effects of Reading Apprenticeship® on adolescent learners. One 
study (Kemple et al., 2008) is a randomized controlled trial that 
meets WWC evidence standards. The remaining 26 studies do 
not meet either WWC evidence standards or eligibility screens. 

Meets evidence standards 

Kemple et al. (2008) conducted a randomized controlled trial 
that examined the effects of Reading Apprenticeship® on 
9th-grade students attending 17 high schools from 10 school 
districts across the United States. Students reading two or more 
years below grade level were randomly assigned to be enrolled 
in Reading Apprenticeship® or to serve as the control group. 

The WWC based its effectiveness rating on findings from two 
cohorts of students who were reading at least two years below 



grade level. Cohort 1 was formed in the 2005/06 school year and 
consisted of 686 ninth-grade students who received Reading 
Apprenticeship® and 454 ninth-grade students in the control 
group who did not receive Reading Apprenticeship®. Cohort 2 
was formed in the 2006/07 school year and consisted of 645 
ninth-grade students who received Reading Apprenticeship® 
and 470 ninth-grade students in the control group who did not 
receive Reading Apprenticeship®. 5 The study reported students’ 
outcomes after 7.5 to 9 months of program implementation. 

Extent of evidence 

The WWC categorizes the extent of evidence in each domain as 
small or medium to large (see the WWC Procedures and Stan- 
dards Handbook, Appendix G). The extent of evidence takes into 
account the number of studies and the total sample size across 



5. Findings for Cohort 2 ninth-grade students are reported in Corrin et al. (2008). See References section for more information. 
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Research (continued) 



Effectiveness 



The WWC found Reading 
Apprenticeship i® to have 
potentially positive effects 
on comprehension for 
adolescent learners 



the studies that meet WWC evidence standards with or without 
reservations. 6 

The WWC considers the extent of evidence for Reading 
Apprenticeship® to be small for the comprehension domain for 



adolescent learners. No studies that meet WWC evidence stan- 
dards with or without reservations examined the effectiveness of 
Reading Apprenticeship® on adolescent learners in the alphabet- 
ics, reading fluency, or general literacy achievement domains. 



Findings 

The WWC review of Adolescent Literacy interventions addresses 
student outcomes in four domains: alphabetics, reading fluency, 
comprehension, and general literacy achievement. The study 
included in this report covers one domain, comprehension, 
which includes two constructs: reading comprehension and 
vocabulary development. The findings below present the 
authors’ estimates and WWC-calculated estimates of the size 
and the statistical significance of the effects of Reading Appren- 
ticeship® on adolescent learners. 7 

Comprehension. Kemple et al. (2008) did not find statistically 
significant effects of Reading Apprenticeship® on the reading 
comprehension and vocabulary subtests of the Group Reading 
Assessment and Diagnostic Evaluation (GRADE) for Cohort 
1 students. For Cohort 2 students, the authors did not find 
statistically significant effects of Reading Apprenticeship® on the 



GRADE vocabulary subtest, but found, and the WWC confirmed, 
a statistically significant positive effect of Reading Apprentice- 
ship® on the GRADE reading comprehension subtest. Thus, 
for the comprehension domain, one study showed statistically 
significant positive effects of Reading Apprenticeship®. 

Rating of effectiveness 

The WWC rates the effects of an intervention in a given outcome 
domain as positive, potentially positive, mixed, no discernible 
effects, potentially negative, or negative. The rating of effectiveness 
takes into account four factors: the quality of the research design, 
the statistical significance of the findings, the size of the difference 
between participants in the intervention and the comparison condi- 
tions, and the consistency in findings across studies (see the WWC 
Procedures and Standards Handbook, Appendix E). 



Improvement index 

The WWC computes an improvement index for each individual 
finding. In addition, within each outcome domain, the WWC 
computes an average improvement index for each study and an 
average improvement index across studies (see WWC Proce- 
dures and Standards Handbook, Appendix F). The improvement 
index represents the difference between the percentile rank of 
the average student in the intervention condition and the per- 
centile rank of the average student in the comparison condition. 



Unlike the rating of effectiveness, the improvement index is 
entirely based on the size of the effect, regardless of the statisti- 
cal significance of the effect, the study design, or the analysis. 
The improvement index can take on values between -50 and 
+50, with positive numbers denoting favorable results for the 
intervention group. 

The average improvement index for comprehension is +2 per- 
centile points, with a range of -2 to +5 percentile points across 
findings from one study. 



6. The extent of evidence categorization was developed to tell readers how much evidence was used to determine the intervention rating, focusing on the 
number and size of studies. Additional factors associated with a related concept— external validity, such as the students’ demographics and the types 
of settings in which studies took place— are not taken into account for the categorization. Information about how the extent of evidence rating was 
determined for Reading Apprenticeship ® is in Appendix A5. 

7. The level of statistical significance was reported by the study authors or, when necessary, calculated by the WWC to correct for clustering within 
classrooms or schools and for multiple comparisons. For the formulas the WWC used to calculate the statistical significance, see WWC Procedures and 
Standards Handbook, Appendix C for clustering and WWC Procedures and Standards Handbook, Appendix D for multiple comparisons. In the case of 
Kemple et al. (2008), a correction for multiple comparisons was needed, so the significance levels may differ from those reported in the original study. 
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Summary 

The WWC reviewed 27 studies on Reading Apprenticeship ® 
for adolescent learners. 8 One of these studies meets WWC 
evidence standards; the remaining 26 studies do not meet either 



WWC evidence standards or eligibility screens. Based on one 
study, the WWC found potentially positive effects on compre- 
hension for adolescent learners. The conclusions presented in 
this report may change as new research emerges. 
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report no. 2009-4036). Washington, DC: Institute of Educa- 
tion Sciences, National Center for Education Evaluation 
and Regional Assistance. 
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protocol or do not meet WWC evidence standards 
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primary analysis of the effectiveness of an intervention, such 
as a meta-analysis or research literature review. 

Chilla, N. (2006). AYP gains for below-proficient readers in a top- 
ranking high school: Woodrow Wilson Senior High School, 
District of Columbia School System. San Francisco, CA: 
WestEd. The study is ineligible for review because it does not 
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Cziko, C. (1998). Reading happens in your mind, not in your 
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or research literature review. 
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Greenleaf, C., Brown, W., & Litman, C. (2004). Apprenticing 
urban youth to science literacy. In D. S. Strickland & D. E. 
Alvermann (Eds.), Bridging the gap: Improving literacy learning 
for preadolescent and adolescent learners in grades 4-12 
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The study is ineligible for review because it does not use a 
comparison group. 

Greenleaf, C., Hanson, T., Herman, J., Litman, C., Madden, 

S., Rosen, R., et al. (2009). Integrating literacy and science 
instruction in high school biology: Impact on teacher practice, 



8. One single-case design study identified but not included in this review because the WWC did not have standards for reviewing single-case design 
studies. 
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Appendix A1 Study characteristics: Kemple et al., 2008 



Characteristic 


Description 


Study citation 


Kemple, J. J., Corrin, W., Nelson, E., Salinger, I, Herrmann, S., & Drummond, K. (2008). The Enhanced Reading Opportunities study: Early impact and implementation findings 
(NCEE 2008-4015). Washington, DC: Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. 

Additional source: 

Corrin, W., Somers, M., Kemple, J. J., Nelson, E., & Sepanik, S. (2008). The Enhanced Reading Opportunities study: Findings from the second year of implementation (NCEE 
report no. 2009-4036). Washington, DC: Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. 


Participants 


The study is a randomized controlled trial that used a two-stage random assignment design. First, 34 eligible schools were randomly assigned to implement one of two 
supplemental literacy programs: 17 schools were assigned to Reading Apprenticeship!® and 17 to Xtreme Reading , 1 Second, in each of the 17 schools assigned to Reading 
Apprenticeship i®, 9th-grade students were randomly assigned to Reading Apprenticeship!® or to the control group. Eligible students were defined as those who were reading at 
least two years below grade level. The study includes two cohorts of 9th-grade students: Cohort 1 was formed in the 2005/06 school year and consisted of 686 ninth-grade 
students who received Reading Apprenticeship!® and 454 ninth-grade students in the control group who did not. Cohort 2 was formed in the 2006/07 school year and 
consisted of 645 ninth-grade students who received Reading Apprenticeship!® and 470 ninth-grade students in the control group who did not. Overall and differential attrition 
rates of student attrition were low for Cohort 1 (30% and 6%, respectively) and Cohort 2 (36% and 3%, respectively). 


Setting 


The study was conducted in 17 schools located in 10 school districts across the United States. 


Intervention 


The intervention group received the Reading Apprenticeship!® Academic Literacy course as a supplemental intervention that replaced a 9th-grade elective class, rather than a 
core academic class. Therefore, students in the intervention group continued to attend their regular English language arts classes. Reading Apprenticeship!® Academic Literacy 
includes a detailed curriculum that is guided by the concept of "flexible fidelity” (i.e., teachers have flexibility in how they implement the curriculum in their day-to-day activi- 
ties). Students attended Reading Apprenticeship!® classes for about 11 hours per month, on average. The study reported students' outcomes after 7.5 to 9 months of program 
implementation. 


Comparison 


The control group received the standard instruction provided in the regular school curriculum and continued their participation in any regularly scheduled elective class (such 
as career/technical education, art, physical education, health, or foreign language). 


Primary outcomes 
and measurement 


For both the pretest and posttest, students took the reading comprehension subtest of the Group Reading Assessment and Diagnostic Evaluation (GRADE). For the posttest, 
students also took the GRADE vocabulary subtest. For a more detailed description of these outcome measures, see Appendix A2. 


Staff/teacher training 


Teachers took part in professional development activities prior to the start of the school year and on an ongoing basis throughout the school year. Training included one 5-day 
summer training institute as well as two 2-day booster sessions during the school year. They also received three 2-day coaching visits during the year and had access to a 
special online listserv. 



1. Comparison between the Xtreme Reading group and the control group is outside the scope of this review. The study also presented impact findings for the subgroups of students defined by 
their baseline reading comprehension test scores, whether they were overage for the ninth grade, and whether a language other than English was spoken in their homes. These subgroup analy- 
ses were presented for the combined intervention group that included both literacy programs: Xtreme Reading and Reading Apprenticeship®. These findings are also outside the scope of this 
review because the measures of effectiveness cannot be attributed solely to Reading Apprenticeship®. 
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Appendix A2 Outcome measures for the comprehension domain 



Outcome measure 


Description 


Reading comprehension construct 


Group Reading Assessment 
and Diagnostic Evaluation 
(GRADE): reading 
comprehension subtest 


GRADE is a norm-referenced reading assessment that can be used with students in pre-K through adulthood. The GRADE has four subtests: (1) vocabulary, (2) sentence 
comprehension, (3) passage comprehension, and (4) listening comprehension. The reading comprehension subtest score is a composite of the sentence comprehension and 
passage comprehension subtest scores (as cited in Kemple et al., 2008). 


Vocabulary development construct 


GRADE: 

vocabulary subtest 


GRADE is a norm-referenced reading assessment that can be used with students in pre-K through adulthood. The vocabulary subtest assesses students’ decoding and 
vocabulary knowledge (as cited in Kemple et al., 2008). 
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